Color transformation for improved traffic sign detection

This paper considers large scale traffic sign detection on a dataset consisting of high-resolution street-level panoramic photographs. Traffic signs are automatically detected and classified with a set of state-of-the-art algorithms. We introduce a color transformation to extend a Histogram of Oriented Gradients (HOG) based detection algorithm to further improve the performance. This transformation uses a specific set of reference colors that aligns with traffic sign characteristics, and measures the distance of each pixel to these reference colors. This results in an improved consistency on the gradients at the outer edge of the traffic sign. In an experiment with 33, 400 panoramic images, the number of misdetections decreased by 53.6% and 51.4% for red/blue circular signs, and by 19.6% and 28.4% for yellow speed bump signs, measured at a realistic detector operating point

Exploiting street-level panoramic images for large-scale automated surveying of traffic sign

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively

A semi-automatic traffic sign detection, classification, and positioning system

The availability of large-scale databases containing street-level panoramic images offers the possibility to perform semi-automatic surveying of real-world objects such as traffic signs. These inventories can be performed significantly more efficiently than using conventional methods. Governmental agencies are interested in these inventories for maintenance and safety reasons. This paper introduces a complete semi-automatic traffic sign inventory system. The system consists of several components. First, a detection algorithm locates the 2D position of the traffic signs in the panoramic images. Second, a classification algorithm is used to identify the traffic sign. Third, the 3D position of the traffic sign is calculated using the GPS position of the photographs. Finally, the results are listed in a table for quick inspection and are also visualized in a web browser

Exploiting street-level panoramic images for large-scale automated surveying of traffic sign

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively

Mutation detection for inventories of traffic signs from street-level panoramic images

Road safety is positively influenced by both adequate placement and optimal visibility of traffic signs. As their visibility degrades over time due to e.g. aging, vandalism, accidents and vegetation coverage, up-to-date inven­tories of traffic signs are highly attractive for preserving a high road safety. These inventories are performed in a semi-automatic fashion from street-level panoramic images, exploiting object detection and classification tech­niques. Next to performing inventories from scratch, these systems are also exploited for the efficient retrieval of situation changes by comparing the outcome of the automated system to a baseline inventory (e.g. performed in a previous year). This allows for specific manual interactions to the found changes, while skipping all unchanged situations, thereby resulting in a large efficiency gain. This work describes such a mutation detection approach, with special attention to re-identifying previously found signs. Preliminary results on a geographical area con­taining about 425 km of road show that 91.3% of the unchanged signs are re-identified, while the amount of found differences equals about 35% of the number of baseline signs. From these differences, about 50% correspond to physically changed traffic signs, next to false detections, misclassifications and missed signs. As a bonus, our approach directly results in the changed situations, which is beneficial for road sign maintenance

Applying feature selection techniques for visual dictionary creation in object classification

This paper introduces improved methods for visual dictionary creation in an object classification system. In literature, the visual dictionary is often created from a large candidate set of features by random selection or by a clustering algorithm. We apply techniques from feature selection literature to create a more optimal visual dictionary and contribute with a novel feature selection algorithm. As a second step, feature extraction techniques for creating the candidate set are investigated. Subsequently, the size of the candidate set is varied. It was found that the exploitation of feature selection techniques gives a clear improvement of 2-5% in classification rate at no additional computational cost in normal system operation. The proposed algorithm called extremal optimization, outperforms state-of-the-art algorithms. The paper discloses results on candidate set creation using interest point operators. As a general bonus, the evaluated feature selection techniques are generally applicable to any problem that uses a dictionary of features, as typically applied in the object recognition domain

System for semi-automated surveying of street-lighting poles from street-level panoramic images

Accurate and up-to-date inventories of lighting poles are of interest to energy companies, beneficial for the transition to energy-efficient lighting and may contribute to a more adequate lighting of streets. This potentially improves social security and reduces crime and vandalism during nighttime. This paper describes a system for automated surveying of lighting poles from street-level panoramic images. The system consists of two independent detectors, focusing at the detection of the pole itself and at the detection of a specific lighting fixture type. Both follow the same approach, and start with detection of the feature of interest (pole or fixture) within the individual images, followed by a multi-view analysis to retrieve the real-world coordinates of the poles. Afterwards, the detection output of both algorithms is merged. Large-scale validations, covering about 135 km of road, show that over 91% of the lighting poles is found, while the precision remains above 50%. When applying this system in a semi-automated fashion, high-quality inventories can be created up to 5 times more efficiently compared to manually surveying all poles from the images

Robust classification of traffic signs using multi-view cues

Traffic sign inventories are created for road safety and maintenance based on street-level panoramic images. Due to the large capturing interval, large viewpoint deviations between the different capturings occur. These viewpoint variations complicate the classification procedure, which aims at the selection of the correct sign type, out of a high number of nearly similar sign types, typically resulting in misclassifications. This paper describes a novel approach for incorporating viewpoint information to the classification procedure, where the sign orientation is estimated based on dense matching. Afterwards, each sample is corrected to a frontal viewpoint, which is then classified. Finally, the sign type is obtained by weighted voting. Large-scale experiments including 2, 224 traffic signs show that this approach reduces the misclassification rate by about 33% compared to the single-view case

Robust classification system with reliability prediction for semi-automatic traffic-sign inventory systems

Inventories of traffic signs are acquired from street-level images in a semi-automated fashion, employing object detection and classification techniques. This is a challenging task, as signs are captured from different viewpoints and under various weather conditions. Furthermore, many similar signs exist, only differing in minor details, and moreover, sign-like objects occur frequently. Consequently, current state-of-the-art systems are unable to reach the required quality level, implying the need for manual corrections. This involves checking all classification results to correct the small minority of misclassifications. This paper presents a classification approach aiming at both high recognition scores and predicting the reliability of the classification output, enabling selective manual intervention. Two reliability prediction methods are compared, analyzing either the classifier scores, or matching the input samples with predefined templates. Large-scale experiments performed for three sign classes, each containing numerous sign types, show that over 80% of the correctly classified results can be marked as reliable, while not marking any misclassifications as reliable. Hence, our research shows that a reliable prediction is possible and that manual invention can be concentrated to the about 25% remaining samples only. Overall, 92.7% of the 8, 159 signs are classified correctly